Simple and Effective Stochastic Neural Networks

نویسندگان

چکیده

Stochastic neural networks (SNNs) are currently topical, with several paradigms being actively investigated including dropout, Bayesian networks, variational information bottleneck (VIB) and noise regularized learning. These network variants impact major considerations, generalization, compression, robustness against adversarial attack label noise, model calibration. However, many existing complicated expensive to train, and/or only address one or two of these practical considerations. In this paper we propose a simple effective stochastic (SE-SNN) architecture for discriminative learning by directly modeling activation uncertainty encouraging high variability. Compared SNNs, our SE-SNN is simpler implement faster produces state the art results on compression pruning, defense,

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

A simple chaotic neuron model: stochastic behavior of neural networks.

We have briefly reviewed the occurrence of the post-synaptic potentials between neurons, the relationship between EEG and neuron dynamics, as well as methods of signal analysis. We propose a simple stochastic model representing electrical activity of neuronal systems. The model is constructed using the Monte Carlo simulation technique. The results yielded EEG-like signals with their phase portr...

متن کامل

Simple Stochastic Temporal Constraint Networks

Many artificial intelligence tasks (e.g., planning, situation assessment, scheduling) require reasoning about events in time. Temporal constraint networks offer an elegant and often computationally efficient framework for formulation and solution of such temporal reasoning tasks. Temporal data and knowledge available in some domains is necessarily imprecise e.g., as a result of measurement erro...

متن کامل

Stochastic Hopfield neural networks

Hopfield (1984 Proc. Natl Acad. Sci. USA 81 3088–92) showed that the time evolution of a symmetric neural network is a motion in state space that seeks out minima in the system energy (i.e. the limit set of the system). In practice, a neural network is often subject to environmental noise. It is therefore useful and interesting to find out whether the system still approaches some limit set unde...

متن کامل

rodbar dam slope stability analysis using neural networks

در این تحقیق شبکه عصبی مصنوعی برای پیش بینی مقادیر ضریب اطمینان و فاکتور ایمنی بحرانی سدهای خاکی ناهمگن ضمن در نظر گرفتن تاثیر نیروی اینرسی زلزله ارائه شده است. ورودی های مدل شامل ارتفاع سد و زاویه شیب بالا دست، ضریب زلزله، ارتفاع آب، پارامترهای مقاومتی هسته و پوسته و خروجی های آن شامل ضریب اطمینان می شود. مهمترین پارامتر مورد نظر در تحلیل پایداری شیب، بدست آوردن فاکتور ایمنی است. در این تحقیق ...

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence

سال: 2021

ISSN: ['2159-5399', '2374-3468']

DOI: https://doi.org/10.1609/aaai.v35i4.16436